Papers with production system
Diversifying Reply Suggestions Using a Matching-Conditional Variational Autoencoder (N19-2)
Copied to clipboard
| Challenge: | Automated reply suggestions (SR) are becoming common in many popular applications such as Gmail (2016) . |
| Approach: | They propose a constrained-sampling approach to make the variational inference efficient for a commercial instant-messaging system. |
| Outcome: | The proposed model increases diversity without losing relevance in offline experiments. |
An End-to-End Dialogue Summarization System for Sales Calls (2022.naacl-industry)
Copied to clipboard
| Challenge: | Summarizing sales calls is a routine task performed manually by salespeople. |
| Approach: | They propose a production system which combines generative models fine-tuned for customer-agent setting, with a human-in-the-loop user experience for an interactive summary curation process. |
| Outcome: | The proposed system can handle training data scarcity and privacy constraints in an industrial setting. |
Entity resolution for noisy ASR transcripts (D19-3)
Copied to clipboard
| Challenge: | Domain-agnostic Automatic Speech Recognition systems often mistranscribe domain-specific words and phrases. |
| Approach: | They propose a method for handling ASR errors in named entities, specifically person names, for a voice-based collaboration assistant. |
| Outcome: | The proposed method improves accuracy by 40.8% on a voice-based collaboration assistant. |
Language Scaling for Universal Suggested Replies Model (2021.naacl-industry)
Copied to clipboard
Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang
| Challenge: | We consider scaling automated suggested replies (SR) to multiple languages for a commercial email application. |
| Approach: | They propose a multi-lingual multi-task continual learning framework with auxiliary tasks and language adapters to train universal language representation across regions. |
| Outcome: | The proposed model reduces catastrophic forgetting and improves cross-lingual transfer across languages while reducing training costs. |
Graph-Linguistic Fusion: Using Language Models for Wikidata Vandalism Detection (2025.acl-industry)
Copied to clipboard
| Challenge: | Wikidata is a large open-source structured knowledge base that is used by search engines, robots, and scripts. |
| Approach: | They propose a vandalism detection system for Wikidata that converts edits into a single space using a method called Graph2Text. |
| Outcome: | The proposed system outperforms the current production system and is released under an open license. |
News Risk Alerting System (NRAS): A Data-Driven LLM Approach to Proactive Credit Risk Monitoring (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Credit risk monitoring is an essential process for financial institutions to evaluate the creditworthiness of borrowing entities. |
| Approach: | They propose a system which proactively alerts Credit Officers to credit-relevant news events . the system has been deployed for nearly three years and has an estimated precision of 77% . |
| Outcome: | The new system has alerted Credit Officers to over 2700 credit-relevant events with an estimated precision of 77%. |
Learning Geolocations for Cold-Start and Hard-to-Resolve Addresses via Deep Metric Learning (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing systems for learning geolocation fail to cater to a significant fraction of addresses which are new in the system and have inaccurate or missing building level information. |
| Approach: | They propose a framework to resolve addresses to a shallower granularity termed neighbourhood . they propose 'deep metric learning' model to encode geospatial semantics in address embeddings . |
| Outcome: | The proposed framework reduces delivery defects and delivery defects in India and the United Arab Emirates. |
Scalable and Safe Remediation of Defective Actions in Self-Learning Conversational Systems (2023.acl-industry)
Copied to clipboard
| Challenge: | Off-Policy reinforcement learning has been used to improve conversational AIs, but in large-scale commercial environments it is challenging to balance between policy improvements and experience continuity. |
| Approach: | They propose to curate and leverage regression incident reports to validate, safe-guard, and improve policies prior to the online deployment. |
| Outcome: | The proposed method validates, safe-guards, and improves policies prior to the online deployment. |
Error Detection in Large-Scale Natural Language Understanding Systems Using Transformer Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Large-scale conversational assistants process every utterance using multiple models for domain, intent and named entity recognition. |
| Approach: | They combine utterance encodings from a RoBERTa model with the Nbest hypothesis produced by the production system to detect domain classification errors. |
| Outcome: | The proposed approach outperforms bi-LSTM models and a standalone model by 2.2% to 32.2% by ensembling multiple models. |
IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing (2024.emnlp-industry)
Copied to clipboard
Kang Chen, Qing Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, Jufeng Chen
| Challenge: | Unlike professional Business-to-Consumer (B2C) e-commerce platforms, consumer-to consumer (C2C), is mainly targeting individual sellers. |
| Approach: | They develop an intelligent product listing tool that generates product descriptions using various product attributes such as category, brand, color, condition, etc. |
| Outcome: | The proposed tool outperforms the base model in domain-specific tasks while producing less hallucination. |
EVE: A Domain-Specific LLM Framework for Earth Intelligence (2026.acl-industry)
Copied to clipboard
Àlex R. Atrio, Antonio Lopez, Jino Rohit, Yassine El Ouahidi, Marcello Politi, Vijayasri Iyer, Umar Jamil, Sébastien Bratières, Nicolas Longépé
| Challenge: | Earth Virtual Expert (EVE) is the first open-source, end-to-end initiative for developing and deploying domain-specialized LLMs for Earth Intelligence. |
| Approach: | They introduce Earth Virtual Expert, an open-source initiative for developing and deploying domain-specialized LLMs for Earth Intelligence. |
| Outcome: | The proposed model outperforms existing models on Earth Observation and Earth Sciences benchmarks while maintaining general capabilities. |
Faithful Model Evaluation for Model-Based Metrics (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies do not consider variance change due to metric model errors, which can lead to wrong conclusions. |
| Approach: | They establish the mathematical foundation of significance testing for model-based metrics . they show that metric errors can change the conclusions in certain experiments . |
| Outcome: | The proposed method can be used to derive accurate conclusions using model evaluations. |